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Key points: 

• The national debate over the efficacy of state-level exams has largely ignored whether 
assessments in civics enhance democratic education, but a 2012 national survey of 18-24-year- 
olds substantiates the hypothesis that civics assessments matter for civic education. 

• While civics assessments required for high school graduation do not appear to influence voter 
turnout or party identification, they do lead to greater civic knowledge in youths, with the greatest 
gains among African Americans, Hispanics, and immigrants — especially Hispanic immigrants. 

• Future research should specifically address why required civics assessments are effective, what 
happens to civic knowledge when states adopt or eliminate their civics assessments, and whether 
the behavior of students, teachers, and administrators changes when a new civics assessment is 
introduced. 


Executive Summary 

In spite of the national debate over the efficacy of state-level exams, whether assessments in 
civics enhance democratic education remains largely unexamined. This paper uses a large 2012 
national survey of 1 8-24-year-olds to examine the potential effect of civics assessments on civic 
outcomes. In doing so, it strives to answer three questions: 

1. Do civics assessments matter? 

Yes, but only assessments that are required for high school graduation (that is, “consequential 
civics assessments”). 


2. For what outcomes do civics assessments matter? 


Consequential civics assessments lead to greater civic knowledge but do not foster greater voter 
participation. Nor do they influence partisan or ideological leanings. 

3. For whom do civics assessments matter most? 

Consequential civics assessments lead to the greatest gains in civic knowledge among African 
American. Hispanic, and immigrant youth — especially Hispanic immigrants. These assessments, 
however, do not have a systematic effect on the methods used for teaching civics. Future research 
should therefore focus on how students, teachers, and administrators adapt to the presence of 
consequential assessments in civic education. 


Amidst the cut and thrust of debate over American education policy, the civic dimension of our 
public education system often gets short shrift. The relative inattention to civic education is both 
lamentable and ironic given that civic education — that is, preparation for democratic 
citizenship — was originally a primary objective of the public school system. 

This inattention means that civic education is often relegated to the sidelines in the discussion of 
concrete initiatives within education policy, even though all states have developed standards for 
civics (or related subjects, like social studies) and 40 states require students to complete at least 
one civics course. 1 Accountability, and assessment specifically, figure among the most 
contentious issues within education policy, yet civics has largely been left out of the discussion. 

The debate over assessment centers on fundamental questions such as how students should be 
assessed, particularly at the state level, and whether assessment results in accountability and thus 
higher academic performance. Since the passage of the federal No Child Left Behind Act 
(NCLB ) — and in some states, since well before that — students have taken statewide standardized 
exams designed to ensure their competence in key subjects. In most states, civics is not included 
within the statewide testing regimen, and the general trend has been for states to drop civics 
assessments. 2 In particular, NCLB does not include civics among the subjects in which students 
must be tested. As a result, little is known about whether statewide assessments in civics make 
any difference. 
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For other subjects, notably math and reading, some evidence suggests that statewide 
assessments — typically, exams — boost student performance . 3 Fiowever, this finding remains 
contentious, as some studies conclude that there are deleterious consequences of statewide 
assessment and accountability standards . 4 Nonetheless, proponents of assessment argue that well- 
designed accountability measures spur students, teachers, and administrators alike to adopt the 
best practices to boost students’ academic performance. 

If the assessments are indeed designed well, then students — and, relatedly, their teachers and 
administrators — can be appropriately evaluated by the exams. By this logic, teacher evaluation 
would not require monitoring teachers’ specific pedagogical practices but instead only the 
outcomes of their students’ performance. That is, the focus would be on the ends and not the 
means, leaving teachers to determine the optimal methods to achieve the objective of their 
students’ strong performance on assessments. 

Similarly, administrators are left to decide which instructors should be teaching civics. 

Obviously, in practice there are many factors outside of the classroom that affect students’ 
performance, which is one reason why assessment is a controversial topic. Should teachers only 
be evaluated based on changes in their students’ performance (that is, value-added analysis)? 
Should an assessment take into account possible confounding factors, such as the socioeconomic 
status of the students? What does a fair and informative assessment entail? 

While contentious, the debate over assessments has the virtue of being grounded in empirical 
analysis. A huge volume of data exists on those subjects — primarily math and reading — that has 
been subjected to testing. In other words, there can be a debate because there is something to 
debate. 


The Who, What, and Why of Civics Assessments 

In shaip contrast to the large literature on assessments’ effects regarding math and reading, very 
few studies have examined what effect, if any, statewide assessments in civics and related 
subjects have on civic education. And to the extent that there has been any research on state-level 
policies regarding civic education — including but not limited to assessments — these studies have 
concluded that these policies have no discernible effect on civic attitudes and behavior . 5 Yet these 
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studies are few, so notwithstanding their null findings, this paper proceeds from the premise that 
the issue is not yet settled and thus poses the question anew: do civics assessments matter for 
civic education? 


The current research into this question resembles past literature on the more general question of 
whether civics courses have any effect on young people’s beliefs and actions. For decades, there 
was a virtual consensus that civics courses had little effect, based largely on research done in the 
1960s. More recently, however, a growing body of research has found that civics courses can 
have an effect, even if those effects arc relatively modest. 6 A common explanation for the limited 
effect is that civics, unlike many other subjects, can be learned outside the classroom by, for 
example, conversations with parents or following the news. 

An alternative explanation, however, could be that the stakes are low for both students and 
teachers when it comes to the subject of civics. Absent an external assessment of what students 
are learning in civics courses, perhaps both students and teachers do not prioritize the subject. If 
so, we should not be surprised that civics courses have little measurable impact on students. 

Civics courses consequently have little impact. It would stand to reason that having a civics 
assessment increases the seriousness with which teachers and students treat civics, leading to 
measurable results. That is, an assessment in civics means better performance in civic outcomes. 
(Later, I address in detail what civic outcomes should be measured.) 

Scholars, policymakers, and teachers who focus on civic education have debated the utility of 
civics assessments, specifically whether civics should be included among the subjects examined 
within accountability systems established under NCLB . On the one hand, there are those who 
worry that the nature of civics does not lend itself to standardized assessment. These opponents to 
civics testing worry that a systematic form of assessment will narrow civic education to only what 
appears on a standardized exam. In other words, they arc concerned that civic education will 
merely become an exercise in teaching to the test — specifically, a test that does not cover the full 
range of what successful civic education should entail. 

On the other hand, some civic educators worry that the absence of civics assessment devalues its 
significance as a subject, which means that it receives too little time, attention, and resources. In a 
2010 AEI Program on American Citizenship survey of high school social studies teachers, an 
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overwhelming majority — 93 percent — said they would prefer that civics (social studies) be 
subject to regular testing to ensure that it was not ignored, even though many are also critical of 
the testing required by NCLB. 7 

Much of the debate over including civics as a subject for assessment rests on what civic education 
should encompass. Should it center on knowledge? If so, knowledge of what? Should civic 
education be defined to include the skills necessary for participation in the public square, such as 
public speaking and running meetings? 8 Should civic educators teach students to have certain 
dispositions, such as tolerance for differing viewpoints and a sense of civic responsibility? 9 If so, 
should these dispositions be evaluated? 

While an interesting and spirited debate centers on the question of what civic education should 
include, my analysis focuses on the civics outcome with the broadest consensus: knowledge. The 
National Assessment of Educational Progress (NAEP) Civics Exam, for example, includes only 
factual questions about the how, what, when, and why of American government and does not 
attempt to examine civics skills or dispositions. 

Recently, Peter Levine and Richard Nierni, two well-known scholars of civic education, called for 
renewed attention to the sort of knowledge we should expect young people to have. They 
suggested that civic educators should focus on ensuring that their students are fluent in current 
events, as this is more likely to spur their engagement in the political process than the traditional 
emphasis on constitutional basics (for example, understanding how a bill becomes a law or how 
the Constitution can be amended). 10 The data employed in this paper include both types of 
knowledge, namely that of the contemporary political landscape and some constitutional 
fundamentals. 

Decades of research support an emphasis on civic knowledge, as numerous studies have shown 
that knowledge is a precursor to civic engagement. 11 But even absent a familiarity with this 
research, it seems self-evident that a representative democracy rests on an informed electorate. 
Social scientists debate the level of knowledge needed for democracy to function properly, but no 
one seriously questions the intrinsic need for an electorate comprised of voters who have at least a 
modicum of knowledge about how their government works. Furthermore, it seems sensible that if 
a course of study has any effect, it would be to increase knowledge within that subject area. 
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While the analysis to follow will center on knowledge as the most plausible and defensible civic 
outcome to be measured, it will also examine other potential civic outcomes that previous 
research has suggested fall within the purview of civic education — specifically, whether statewide 
civics assessments have an effect on young people’s political participation. (Are young people 
who took civics assessments in high school more likely to turn out to vote?) 

And even though civic education is ostensibly nonpartisan, the heated debates over civics 
standards, and over civics’ close cousin history, suggest a concern that civics entails political 
indoctrination. 12 Consequently, my analysis will also examine whether civics assessments have 
an effect on either party identification or political ideology. 

In sum, assuming that civics assessments do have an effect on outcomes, in the next part of the 
paper I examine the specific outcomes for which they matter, hypothesizing that assessments 
have a bearing on political knowledge. The analysis then turns to ask: For whom do civics 
assessments matter most? Do they have the same effect across all student groups, or do they 
matter more for some students? 

The fact that US public schools were created for civic purposes underscores that the nation’s 
education system has long served as the proverbial melting pot. Public schools bring together 
young people of many different ethnicities to forge a common identity and to school them in 
democratic virtues, such as respect for the symbols and institutions of government, a sense of 
civic responsibility, and an appreciation for individual rights. In short, the public schools have 
been a leading institution for creating unurn out of pluribus. One important aspect of this great 
melting together is teaching young immigrants and children of immigrants about America’s 
system of government, thus preparing — and motivating — them for participation in the nation’s 
political life. 13 

Similarly, past research on civic education has shown that formal instruction in civics matters 
most among students who are otherwise the least exposed to politics. A seminal study of civic 
education, conducted in 1965, concluded that while civics courses had little effect on students in 
general, they did lead to greater civic knowledge, efficacy, tolerance, and sense of civic duty for 
African American students. 14 In 1965, there was every reason to think that African American 
youth had little exposure to conventional political activity in the home, given that they had been 
raised in an era of overt racism nationwide and formal disenfranchisement in the Jim Crow South. 
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While subsequent research has not always found a greater effect for civic education among 
disadvantaged youth, enough studies have that it remains a viable hypothesis. Specifically, to the 
extent that civics assessments lead to improved civic education, the extant literature suggests that 
assessments would have the biggest effect for immigrants and members of minority groups, two 
categories that obviously overlap. In particular, we should expect that immigrants would benefit 
most from civics instruction, as they are likely to have the least experience with the US political 
system. 

To summarize, past research has suggested that civics assessments — or any state-level policies or 
standards regarding civic education — have no measurable effect on civic outcomes within the 
youth population. This paper revisits that conclusion by focusing on three interrelated questions 
and drawing on the existing literature to suggest what the answers might be. 

1. Do civics assessments matter? 

The general literature on accountability measures suggests that, under some circumstances, 
assessments in subjects other than civics can incentivize educators and students to improve their 
academic performance. It stands to reason that they would have the same effect for civics. 

2 . For what outcomes do civics assessments matter? 

If civics classes — and, specifically, statewide assessments — are to make a difference, they are 
most likely to have an impact on factual knowledge about the political process and institutions. 

3 . For whom do civics assessments matter most? 

Past research suggests that civic education in general matters most for students who have the least 
exposure to the nation’s politics in their homes, which points toward a larger effect for 
immigrants and members of other minority groups. 

I am thus suggesting that past research has cast too wide a net in looking for the effects of state 
standards in civics. The effect of civic education can be subtle and there is no reason to assume 
that it matters equally for everyone in the population. As I explain later, the data do indeed 
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indicate that civics assessments (with academic consequences) appeal - to boost political 
knowledge among young people, especially among African American, Hispanic, and immigrant 
youth. Consequential assessments have the biggest impact of all on Hispanic immigrants. 

After presenting evidence that youth educated in states with a civics assessment have greater 
political knowledge, the remainder of the paper examines why. Does a civics assessment 
systematically affect civics pedagogy? Is civics taught differently (presumably, better) in states 
with an assessment system? The answer is elusive in existing data. However, the absence of a 
single explanation may itself be an important insight into effective civics instruction: perhaps it is 
best to “let a thousand flowers bloom” and leave it to teachers to determine the most effective 
means of achieving the end of ensuring that their students have the necessary level of political 
knowledge to be informed citizens. 

Analysis 

One reason for the lack of attention to civics is the relatively scarce amount of data available on 
the subject, especially when compared to other academic subjects, such as math and reading. 
Fortunately, a recent survey is an important correction to that lacuna. In 2012, the Spencer 
Foundation funded a survey conducted by the Center for Information and Research on Civic 
Learning and Engagement (CIRCLE) and supplemented with data on statewide civics standards 
collected with the support of the Bechtel Foundation. These data are ideally designed to answer 
the three questions I have posed: 15 

1. Do civics assessments matter? 

The Spencer survey was limited to 18-24-year-olds, the group where we are most likely to see an 
effect for civic education. The survey contained a nationally representative sample of 4,483 US 
citizens. In addition to a wide array of information gathered about each of the respondents, 
including detailed demographic data, CIRCLE researchers incorporated data on the 
characteristics of the respondents’ states. This includes standards and requirements for civic 
education in the state where the respondent attended high school. Respondents were also asked 
detailed questions about their experiences in high school, including the general climate of the 
school and the specific pedagogical practices used in civics-related (social studies) classes. 
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2. For what outcomes do civics assessments matter? 


The survey asks a wide array of questions to gauge civic attitudes and behaviors. These include 
voter turnout in the 2012 election and, critically for this study, a battery of questions to measure 
respondents’ civic knowledge. 

3. For whom do civics assessments matter most? 

The survey’s large sample size enables reliable analysis of subgroups within the population, such 
as immigrants and minorities. 

The logic of the analysis is straightforward: do young people who attended high school in states 
with a civics assessment have greater knowledge about government and politics than those who 
did not? 16 The first step in answering this question requires identifying those states with such an 
assessment, and states are either coded as having such an assessment or not. 17 As of 2012, there 
were 2 1 states with such an assessment. This subset of states reflects the diversity of the nation. 
(See table 1.) 

The 21 states are geographically diverse, as they are not concentrated in any single region, and 
they are economically diverse, as their average household income ranges from $45,000 to 
$72,000. Moreover, they are racially diverse, with the states’ Hispanic populations ranging from 
1.2 percent to 46.3 percent and their African American populations ranging from 2.1 percent to 
37 percent. They are also politically diverse, as 8 states went for Obama in 2012 and 13 went for 
Romney. 

Finally, they are educationally diverse, as their NAEP scores range from 25 1 to 269 (national 
average is 263) in reading and from 265 to 289 (national average is 282) in math. Their per-pupil 
expenditures for education range from $7,500 to $19,000 (national average is $1 1,000). While my 
statistical analysis will control systematically for these differences across states, a priori there is 
no obvious confounding factor that characterizes the assessment states. They are a cross-section 
of America. 
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Table 1. Civic Education Requirements by State 


No Civics Requirement 

Iowa 

Civics Course Only 

Arkansas 

Arizona 

Colorado 

Connecticut 

District of Columbia 

Florida 

Hawaii 

Idaho 

Illinois 

Massachusetts 

Maryland 

Maine 

Minnesota 

Montana 

North Carolina 

North Dakota 

Nebraska 

New Hampshire 

New Jersey 

Nevada 

Oregon 

Pennsylvania 

Rhode Island 

South Dakota 

Utah 

Vermont 

Washington 

Wyoming 


Civics Assessment without 
Consequences 

California 

Delaware 

Indiana 

Kansas 

Louisiana 

Michigan 

Missouri 

Oklahoma 

Wisconsin 

West Virginia 


Civics Assessment with 
Consequences 

Alabama 

Georgia 

Kentucky 

Mississippi 

New Mexico 

New York 

Ohio 

South Carolina 
Tennessee 
Texas 
Virginia 


Source: Center for Information & Research on Civic Learning and Engagement, “New CIRCLE Fact Sheet 
Describes State Laws, Standards, and Requirements for K-12 Civics,” 2012, www.civicyouth.org/new- 
circle-fact-sheet-describes-state-laws-standards-and-requirements-for-k-12-civics/ . 


In addition to the presence or absence of a civics assessment, the analysis tests whether the 
number of years civic education is required affects political knowledge, since the critical factor 
might not be the assessment but rather the sheer amount of instruction. Even though a study by 
Deven Carlson found that the number of years of civic instruction has no bearing on civic 
knowledge (as measured by the NAEP) among high school students, this control is nonetheless 
included to see whether quantity of instruction affects the amount of civic knowledge that sticks 
past high school . 18 
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Analysis throughout this section of the paper includes only state-level policies regarding civic 
education. While later analysis incorporates self-reports about the pedagogical techniques 
respondents recall in their high school civics classes, these reports are unavoidably clouded by 
self-selection. That is, people who are civically aware — and thus score higher on an index of 
political knowledge — are also more likely to have taken civics courses in high school, to have 
participated more fully in them, and perhaps even to remember them differently than their less 
civically aware classmates. What one person considers an enlivening classroom discussion might 
bore someone else to tears. 

State-level policies, however, are not subject to self-selection in the same way. Admittedly, it is 
theoretically possible that especially civically engaged families choose to live in states with a 
civics assessment, but that seems implausible. Civics requirements within a state are hardly 
common knowledge and it strains credulity to suggest that a state’s civic education requirements 
are a drawing card for potential move-ins. 

A more plausible concern is that policies across a given state are not implemented consistently or, 
in some districts and schools, even ignored altogether. Given implementation uncertainty, any 
effects observed in this analysis should be considered a lower bound. A medical trial to test a new 
medicine provides a useful analogy. In such a trial, not every patient takes the pill as prescribed — 
either out of forgetfulness or inattention. Thus, medical researchers consider all the subjects 
assigned to take the pill as the “intention to treat” group, whether those subjects actually comply 
with the protocol or not. The analysis then measures whether the pill has an effect, on average, 
within the intention-to-treat group, even though some members of the group may not have 
followed the protocol. 

This is done because, should a drug come to market, it will not always be used correctly and the 
medical community wants to determine the aggregate effect of the experimental treatment. States 
with a civics assessment (or any state-level policy) are like the intention-to-treat group, as not all 
districts and schools will necessarily follow protocol and comply with the rule. 19 The question is 
whether the policy being analyzed has an effect on the aggregate, notwithstanding the 
uncertainties in implementation. 
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The primary outcome of interest is political knowledge, which is measured with an index of six 
questions. While a survey of this type cannot have the same breadth of items as an exam such as 
the NAEP, these questions include some fundamentals of how the American political system 
operates and some rudimentary knowledge about the current political landscape. 20 They arc thus a 
good gauge of informed voting. The questions, which comprise the Civic Knowledge Index, are 
(with correct answers in parentheses): 

1. As far as you know, does the federal government spend more on Social Security or on foreign 
aid? (Social Security) 

2. Would you say that one of the parties is more conservative than the other on the national 
level? (Yes) / Which party is more conservative? (Republican) 

3. Do you happen to know which party had the most members in the House of Representatives 
in Washington before the election this month? (Republican) 

4. How much of a majority is required for the US Senate and House to override a presidential 
veto? (Two-thirds) 

5. Which of the following best describes who is entitled to vote in federal elections? (Multiple- 
choice options were residents, taxpayers, legal residents, citizens. The correct answer is 
citizens.) 

The average score was three out of six correct. Roughly 9 percent got none of them correct, while 
4 percent had a perfect score. The easiest question was whether one party was more conservative 
than the other (69 percent knew that, of whom 76 percent knew it was the Republican Party). The 
most difficult question was whether the US government spends more on Social Security or 
foreign aid. Only 29 percent knew that it is Social Security. 

These questions should not imply that these items arc either necessary or sufficient for informed 
democratic engagement. Rather, the intent is to have a general gauge of respondents’ knowledge 
about politics and government. A more detailed exam would obviously provide more nuanced 
information, differentiating among different domains of knowledge. But such an exam would also 
correlate highly with this short quiz. 
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While blunt, this index is a serviceable measure of general civic knowledge. Indeed, its brevity — 
and thus limited variance — only makes it more difficult to detect a signal amidst the noise. The 
bias of the analysis is thus against finding an effect of statewide assessments, underscoring the 
substantive significance of any effect we might find. 

The analysis proceeds by testing whether 18-24-year-olds who attended high school in states 
with a civics assessment have greater civic knowledge. More formally, the Civic Knowledge 
Index is regressed on civics assessment, but with a host of control variables to ensure that the 
effect of the assessment has been isolated from potentially confounding factors. In other words, 
the analysis is designed to eliminate concern that what appears to be the effect of a civics 
assessment is really the effect of something else that either describes the states that have 
assessments or the people who live in such states. To ensure an apples-to-apples comparison, the 
same variables arc used in the models for each civic outcome discussed in this paper. 

The statistical analysis controls for a host of individual-level characteristics, including race or 
ethnicity (whether the respondent is African American, Asian American, or Hispanic), immigrant 
status, gender, and age. In addition, the model accounts for educational attainment but, because 
many of these respondents are still pursuing their education, it includes both the highest level of 
education attained and whether the respondent is currently enrolled in school. 

Although household income is typically used as a measure of socioeconomic status, income can 
be misleading for this age group. For one, it is not clear whether the appropriate income to be 
measured is respondents’ own or their parents’. In the former case, income often appears 
misleadingly low because respondents are in school or just starting a career. In the latter, they 
often do not know their parents’ income. 

Consequently, intellectual stimulation in the childhood home better captures the presumed 
influence of social class, measured with the deceptively simple question, “When you were 
growing up, about how many books were there in your home?” The choice of answers includes a 
few (0-10), enough to fill one shelf (11-25), enough to fill one bookcase (26-100), or enough to 
fill several bookcases (more than 100). This simple question efficiently yet powerfully measures 
the intellectual climate within the home, which is one important way social class affects all 
educational performance, including civics. 21 
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To ensure that a civics assessment is not masking other state-level characteristics, the analysis 
also accounts for students’ general academic performance in the states where the respondents 
went to high school, measured with mean NAEP scores in math and reading. The model also 
controls for each state’s average per-pupil expenditure and median household income. 

Since the survey was conducted during 2012, a presidential election year, it is also important to 
account for any possible effects of the political environment of the state in which respondents 
were living in 2012. These include the past level of turnout among 18-24-year-olds (measured as 
2010 turnout in the Current Population Survey) and the degree of electoral competition in 2012. 
Historically high levels of turnout and contemporaneous political competition could foster greater 
political engagement and, potentially, knowledge. 22 

Do Civics Assessments Matter? The analysis begins with the fundamental question of whether 
respondents who attended high school in states with a civics assessment have greater civic 
knowledge. In a nutshell, the answer is yes. Students who attended high school in a state without 
a civics assessment scored, on average, 2.8 out of 6.0 on the knowledge scale. Those who were 
educated in states with an assessment scored 3.03, a statistically significant difference (p < .05). 

Importantly, the presence of a civics assessment has the effect, not the number of civics courses 
students are required to take. This finding is notable in light of the high bar - for finding any 
statistically significant effect, given the shortcomings of a six-item knowledge index. However, it 
is just as important to ask whether an increase of 0.23 on the scale has a substantive 
significance — intrinsically a more subjective judgment. One useful benchmark is comparing the 
effect size for a civics assessment to other factors that have a statistically meaningful impact on 
civic knowledge. In making this comparison, the presence of a civics assessment has what can be 
described as a moderate effect, comparable to, for example, being raised in a home with several 
bookcases instead of only one. 

But not all civics assessments are created equal. In some assessment states, the civics exam has 
no bearing on whether a student graduates from high school. In others, graduation does not 
require earning a certain score on the assessment, but the assessment nonetheless counts toward a 
grade in a civics course. In still others, the assessment is a graduation requirement. Essentially, 
the analysis asks whether the stringency of the requirement matters. Do students have higher 
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knowledge when the state-administered civics exam has consequences for graduating high 
school? 

Of the 21 states with a civics assessment, a total of 1 1 either require the exam for graduation (9 
states) or count it toward a final grade in a required civics course (2 states). Because it is not clear 
that the consequences are more or less severe under one system than another, these 1 1 states have 
been combined into a single category and thus treated as functional equivalents. 

Even though they are a subset of a subset, this group of 1 1 states (listed in table 1) is nonetheless 
broadly representative of the United States. Their average median income is $55,000, their 
Hispanic population ranges from 2.7 to 46.3 percent (average is 13 percent), their African 
American population ranges from 2.1 to 37 percent (average is 19 percent), four of them favored 
Obama in 2012, their NAEP math scores range from 265 to 287 (average is 278), and their NAEP 
reading scores run from 251 to 269 (average is 260). 

To answer whether it matters if the civics assessment has academic consequences, the binary 
measure of whether a state has an assessment is replaced with a series of dichotomous variables 
that indicate whether the state falls within one of the following categories: 

1 . No civics requirement or assessment 

2. State requires a civics course but no assessment 

3. State requires a civics course and has an assessment, but the assessment has no bearing on 
graduation or grades 

4. State requires a civics course, has an assessment, and the assessment has consequences. 

Either it counts toward a grade in a civics course that is required for graduation, or it is by 
itself a graduation requirement. 

The results show that consequences matter (details of the statistical models can be found in 
Appendix 1). Whether it is the general population as a whole or a subset (minorities or 
immigrants), civics assessments with academic consequences correspond with greater political 
knowledge. As shown in figure 1, for the population as a whole, respondents who attended high 
school in states that have consequential assessments score about .20 higher on the political 
knowledge scale, essentially the same as in the analysis that only measured whether there is an 
assessment without regard for potential consequences. 23 
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Are these effects substantively significant? As displayed in figure 2, among the general 
population, a consequential assessment has an effect comparable to other factors long known to 
correlate with higher political knowledge, such as education level, home environment, age, and 
gender. 24 


Whatever its substantive size, finding any statistically significant effect for civics assessments is 
remarkable, especially given the noisiness of measuring civic knowledge and, presumably, 
uneven implementation of the state-level requirement. Furthermore, previous research has failed 
to find any effect for any state-level civic education policy on any outcome. 


The analysis, however, shows evidence that even when accounting for a wide variety of 
potentially confounding explanations, the signal of the civics assessment can be detected amidst 
the noise. On the other hand, though, the size of the effect is roughly comparable to other factors 
known to affect political knowledge, suggesting that when we look at the youth population as a 
whole, civics assessments play a meaningful but modest role in fostering political knowledge. So, 
do civics assessments matter? Yes, but — for the population as a whole at least — only modestly. 


Figure 1. Consequential Civics Assessments Correspond with Greater Civic 
Knowledge 



Without an assessment 
■ With an assessment 


Source: Center for Information & Research on Civic Learning and Engagement, “New CIRCLE Fact Sheet 
Describes State Laws, Standards, and Requirements for K-12 Civics,” 2012, www.civicyouth.org/iiew- 
circle-fact-sheet-describes-state-laws-standards-and-requirements-for-k-12-civics/ . 

Note: This figure shows the effect of an assessment with academic consequences and includes statistical 
controls. 


15 


For What Outcomes Do Civics Assessments Matter? Thus far, my discussion of the analysis 
has centered on civic knowledge, the most empirically plausible — and, arguably, normatively 
defensible — outcome of civic education provided by public schools. As noted, however, “civic 
engagement” is multifaceted and it is thus possible that civic education would have an effect on 
more than what people know. Does it also affect how they think and what they do? 


The short answer is no. State assessments, specifically those with academic consequences, had no 
effect on voter turnout in the 2012 election. Nor did they have any effect on young people’s party 
identification or whether their political ideology leans to the left or right. (See Appendix 1 for 
details of this analysis.) 


In short, as hypothesized, only civic knowledge is affected by whether a young person attended 
high school in a state with a consequential civics assessment. Note, however, that the Spencer 
survey did not attempt to measure civic skills or dispositions such as tolerance or efficacy, and so 
it must be left to future research to determine if these are also affected by the presence of a civics 
assessment. 


Figure 2(a). Comparing the Effect of a Consequential Civics Assessment with Other Factors 
Affecting Civic Knowledge (General Population) 
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Source: Center for Information & Research on Civic Learning and Engagement, “New CIRCLE Fact Sheet 
Describes State Laws, Standards, and Requirements for K-12 Civics,” 2012, www.civicyouth.org/new- 
circle-fact-sheet-describes-state-laws-standards-and-requirements-for-k-12-civics/ . 

Note: This figure includes statistical controls. 


16 


Figure 2(b). Comparing the Effect of a Consequential Civics Assessment with Other Factors 
Affecting Civic Knowledge (African Americans) 



Source: Center for Information & Research on Civic Learning and Engagement, “New CIRCLE Fact Sheet 
Describes State Laws, Standards, and Requirements for K-12 Civics,” 2012, www.civicyouth.org/new- 
circle-fact-sheet-describes-state-laws-standards-and-requirements-for-k-12-civics/ . 

Note: This figure includes statistical controls. 


Figure 2(c). Comparing the Effect of a Consequential Civics Assessment with Other 
Factors Affecting Civic Knowledge (All Hispanics) 
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Source: Center for Information & Research on Civic Learning and Engagement, “New CIRCLE Fact Sheet 
Describes State Laws, Standards, and Requirements for K-12 Civics,” 2012, www.civicyouth.org/new- 
circle-fact-sheet-describes-state-laws-standards-and-requirements-for-k-12-civics/ . 

Note: This figure includes statistical controls. 
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Figure 2(d). Comparing the Effect of a Consequential Civics Assessment with Other Factors 

Affecting Civic Knowledge (All Immigrants) 
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Source: Center for Information & Research on Civic Learning and Engagement, “New CIRCLE Fact Sheet 
Describes State Laws, Standards, and Requirements for K-12 Civics,” 2012, www.civicyouth.org/new-circle-fact- 
sheet-describes-state-laws-standards-and-requirements-for-k- 12-civics/ . 

Note: This figure includes statistical controls. 


Figure 2(e). Comparing the Effect of a Consequential Civics Assessment with Other Factors 
Affecting Civic Knowledge (Hispanic Immigrants) 
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Source: Center for Information & Research on Civic Learning and Engagement, “New CIRCLE Fact Sheet 
Describes State Laws, Standards, and Requirements for K-12 Civics,” 2012, w w w . civicvouth. org/new-circle-fact- 
sheet-describes-state-laws-standards-and-requirements-for-k- 12-civics/ . 

Note: This figure includes statistical controls. 
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For Whom Do Civics Assessments Matter Most? Recall the hypothesis that civic education will have 
the biggest impact on both minorities and immigrants, two groups that partially overlap. To test whether 
this is the case, the same model is applied to African Americans, Hispanics, and immigrants. 25 
The results confirm the expectation that civic education in the classroom has a greater effect on African 
American and Hispanic students (as shown in figure 1). For the general population, we have seen that the 
effect of a consequential civics assessment is roughly .20 points on the 6-point knowledge scale, a modest 
magnitude. But it is larger for African Americans and Hispanics (about 0.30 points for each). 

The results also demonstrate that civics assessments with consequences have an especially large effect on 
the civic knowledge of immigrant students (0.45 points). The biggest effect of all is found among 
Hispanic immigrants: 0.78 points. 26 

To provide a sense of substantive significance, figures 2(a) through 2(e) display how the size of the 
assessment effect compares to other factors that correlate with civic knowledge. For African Americans 
and Hispanics, a consequential civics assessment has roughly the same effect as the other variables, as is 
the case for the general population. For all immigrants, a consequential assessment essentially ties with 
age for the biggest effect. And for Hispanic immigrants, attending high school in a state that has a civics 
assessment with academic consequences has the single largest effect. 27 

In other words, here we see evidence that consequential civics assessments have an especially large effect 
among immigrants, and Hispanic immigrants in particular, which is in keeping with the historical role of 
the public school system to facilitate unum among America’s pluribus. 

Do Civics Assessments Affect Methods of Civics Instruction? 

Having established that consequential civics assessments increase civic knowledge — particularly for 
immigrants — the next question is why. Hopefully, the search for a definitive answer to that question will 
guide future research into civic education. The analysis throughout this paper takes a first step in 
determining why consequential assessments lead to greater civic knowledge. 

Finding an answer begins with the fundamental rationale for assessments. Whether for civics or any other 
subject, the justification for a state-imposed assessment is the same. By providing an external measure of 
performance, assessments give students, teachers, and administrators an incentive to ensure that students 
perform well. Administrators will assign better teachers to subjects with an assessment, teachers will 
teach those subjects more effectively, and students will exert more effort to learn those subjects. 28 


19 



Thus, one promising explanation for greater civic knowledge among young people educated in states with 
consequential civics assessments is that civics teachers in those states use different methods in the 
classroom. Critics often describe civic education as dry and boring, relying on lectures and worksheets 
rather than stimulating discussion, meaningful projects, and other engaging pedagogical practices. 29 Past 
research has shown that an engaging classroom — particularly one where teachers encourage the open 
discussion of real-world issues — is an especially effective method for teaching civics, particularly for 
students from disadvantaged backgrounds. 30 

The Spencer survey enables a test of whether classroom practices are affected by the presence of a 
consequential civics assessment. Respondents were asked whether their high school civics classes 
featured any of the following; for each one, they could answer yes or no: 

1. In that (civics) course, did you spend much time discussing current events? 

2. Did teachers encourage students in that class to discuss political and social issues in which people 
have different opinions? 

3. Did you do research on social, political, or community issues for that class? 

4. Did you do a project in the community for that class? 

5. In that class, were you required to keep up with politics or government, either by reading the 
newspaper, watching TV, or going onto the Internet, or not? 31 

Tallying the total positive and negative responses produces a Civics Instruction Index. Then, the same 
model discussed earlier can be used to predict civic outcomes and methods of civics instruction. The 
reasoning is that if these pedagogical practices are the explanation for the connection between 
consequential civics assessments and greater civic knowledge, they should be more common in states 
with such assessments. Likewise, since consequential civics assessments have the biggest impact on 
minorities and immigrants, the analysis examines whether members of these groups are especially likely 
to report that these teaching methods were used in their civics classes. 
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Unlike the analysis of civic knowledge, there is no connection between a consequential civics assessment 
and the frequency of the practices outlined in the five questions, whether they are combined into the 
Civics Instruction Index or analyzed one-by-one (results for the individual item models are not shown but 
available by request). 

If classroom practices do not differ in states with an assessment, what might explain the apparent effect of 
an assessment on civic knowledge? One answer could be that there are other effective pedagogical 
practices employed by teachers in assessment states, in which case researchers should figure out what 
they are. Another explanation is that teachers do not use different methods, but simply use the same 
methods more effectively. Or perhaps the difference lies not in the teachers but in the students: knowing 
that they will have to take an exam with academic consequences, they are motivated to learn more from 
the same methods. 

What does not appeal - to be happening is narrow “teaching to the test” whereby teachers focus their 
instruction on maximizing their students’ scores on one instrument with no regard for broader learning. 32 
Or, if there is teaching to the test, it appears to be working, since the civic knowledge scale used in the 
survey is not calibrated with any state exam(s). 

Perhaps these null results teach a lesson: effective civic education cannot easily be explained by the wide- 
scale adoption of particular classroom practices. Civic educators may be heartened by this conclusion, as 
it might be that the best civic education is what teachers determine it to be based on their particular 
context. Rather than being followers of the cookie-cutter approach, perhaps teachers are more like artisans 
who customize their instruction to what works best for their own students, particularly minority and 
immigrant youth. They just need the right incentive to do so. 

Conclusions and Recommendations 

Three questions have guided the analysis throughout this paper: 

1 . Do civics assessments matter? 

2. For what outcomes do civics assessments matter? 

3. For whom do civics assessments matter most? 
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In brief, the answers arc 1) yes, if the assessments have academic consequences; 2) political knowledge; 
and (c) minorities and immigrants, especially Hispanic immigrants. 

Given these results, it is tempting to conclude that consequential civics assessments should be 
implemented nationwide. However, the rush to such a recommendation is tempered by two facts. First, 
while cross-sectional analysis of this type is informative, it should not be considered dispositive. Second, 
the answer to the question of why assessments have the effect they do remains elusive. Therefore, the 
next steps arc to confirm the assessment effect. 

At minimum, future research should examine what happens to civic knowledge when states adopt or 
eliminate a civics assessment. Similarly, such studies could explore whether the behavior of students, 
teachers, and administrators changes with the introduction of a new assessment. Randomized experiments 
would be even more informative. Does student knowledge increase when students are randomly assigned 
to an external assessment of their civic education? Likewise, randomized experiments can further explore 
which practices do and do not work in civic education, both in the presence and absence of a civics 
assessment. 33 

In conclusion, this paper suggests what can be learned about civic education through necessary 
attention — particularly with appropriate data. With further attention and resources, much more could be 
learned about the effectiveness of civic education, helping ensure that all of America’s youth benefit from 
best practices. The state of citizenship in America would be all the stronger for it. 


David E. Campbell ( Dave_Campbell@nd.edu ) is a professor of political science and director of the 
Rooney Center for the Study of American Democracy at the University of Notre Dame. 


Notes 

1. Center for Information & Research on Civic Learning and Engagement, “New CIRCLE Fact Sheet Describes 
State Laws, Standards, and Requirements for K-12 Civics,” 2012, www.civicyouth.org/new-circle-fact-sheet- 
describes-state-laws-standards-and-requirements-for-k- 12-civics/. 

2. Courses related to civic education go by many names, including civics, government/ American government, and 
social studies (which can also include history, geography, economics, and related subjects). For the sake of brevity, 
this paper refers collectively to all of these subjects as “civics.” According to CIRCLE, in 2001, 34 states regularly 
assessed students’ performance in social studies. By 2006, that had fallen to 21. However, Maryland and Florida 
have defied the general trend and tightened their social studies requirement, as they have recently added a civics 
requirement for high school graduation. Future research will examine the consequences of adding or removing 
civics requirements. See Center for Information & Research on Civic Learning and Engagement, “New CIRCLE 
Fact Sheet.” 


22 



3. Brian A. Jacob, “Accountability, Incentives and Behavior: The Impact of High-Stakes Testing in the Chicago 
Public Schools,” Journal of Public Economics 89 (2005): 761-96; Deven Carlson, Geoffrey D. Borman, and 
Michelle Robinson, “A Multistate District-Level Cluster Randomized Trial of the Impact of Data-Driven Reform 
on Reading and Mathematics Achievement,” Education Evaluation and Policy Analysis 33, no. 3 (2011): 378-98; 
Eric A. Hanushek and Margaret E. Raymond, “Does School Accountability Lead to Improved Student 
Performance?” Journal of Policy Analysis and Management 24, no. 2 (2005): 297-327; Martin Carnoy and Susanna 
Loeb, “Does External Accountability Affect Student Outcomes? A Cross-State Analysis,” Education Evaluation and 
Policy Analysis 24 (2002): 305-31; and Thomas S. Dee and Brian A. Jacob, “The Impact of No Child Left Behind 
on Student Achievement,” Journal of Policy Analysis and Management 30, no. 3 (201 1): 418-46. 

4. Jonathan Supovitz, “Can High Stakes Testing Leverage Educational Improvement? Prospects from the Last 
Decade of Testing and Accountability Reform,” Journal of Educational Change 10 (2009): 211-27; David Hursh, 
“The Growth of High-Stakes Testing in the USA: Accountability, Markets and the Decline of Educational Equality,” 
British Educational Research Journal 31, no. 5 (2005): 605-22; and Sharon L. Nichols, Gene V. Glass, and David 
C. Berliner, “High-Stakes Testing and Student Achievement: Does Accountability Pressure Increase Student 
Learning?” Education Policy Analysis Archives 14, no. 1 (2006): 1-172. 

5. Deven Carlson, “Out of the Classroom and Into the Voting Booth? Analyzing the Effects of Education on 
Political Participation” (dissertation. University of Wisconsin-Madison); Kei Kawashimi and Peter Levine, “Policy 
Effects on Informed Political Engagement,” American Behavioral Scientist 58, no. 5 (2014): 665-88; Lopez et ah, 
“Schools, Education Policy, and the Future of the First Amendment,” Political Communication 26, no. 1 (2009): 84- 
101 . 

6. Kenneth P. Langton and M. Kent Jennings, “Political Socialization and the High School Civics Curriculum in the 
United States,” American Political Science Review 62, no. 3 (1968): 852-67; David E. Campbell, “Voice in the 
Classroom: How an Open Classroom Climate Fosters Political Engagement Among Adolescents.” Political 
Behavior 30, no. 4 (2008): 437-54; ; Carlson, “Out of the Classroom and Into the Voting Booth?”; Richard G. 

Niemi and Jane Junn, Civic Education: What Makes Students Learn (New Haven: Yale University Press, 1998); 
Joseph E. Kahne and Susan E. Sporte, “Developing Citizens: The Impact of Civic Learning Opportunities on 
Students’ Commitment to Civic Participation,” American Educational Research Journal 45, no. 3 (2008): 738-66; 
and Judith Torney-Purta, Carolyn H. Barber, and Britt Wilkenfeld, “Latino Adolescents’ Civic Development in the 
United States: Research Results from the IEA Civic Education Study,” Journal of Youth and Adolescence 36 (2007): 
111-25. 

7. Steve Farkas and Ann Duffet. High Schools, Civics, and Citizenship: What Social Studies Teachers Think and Do 
(AEI Program on American Citizenship, 2009), www.citizenship-aei.org/2010/09/new-aei-report-high-schools- 
civics-and-citizenship/#.VBCt5vldV8F; and Susie Burroughs, Eric Groce, and Mary Lee Webeck, “Social Studies 
Education in the Age of Testing and Accountability,” Educational Measurement: Issues and Practice 24, no. 3 
(2005): 13-20. 

8. Sidney Verba, Kay Lehman Schlozman, and Henry E. Brady, Voice and Equality: Civic Voluntarism in American 
Politics (Cambridge, MA: Harvard University Press, 1995). 

9. David E. Campbell, Why We Vote: How Schools and Communities Shape Our Civic Life (Princeton, NJ: Princeton 
University Press, 2006). 

10. Peter Levine, “Education for a Civil Society,” in Making Civics Count: Citizenship Education for a New 
Generation , eds. David E. Campbell, Meira Levinson, and Frederick M. Hess (Cambridge, MA: Harvard Education 
Press), 37-56; Niemi and Junn, Civic Education: What Makes Students Learn. 

1 1 . Michael X. Delli Carpini and Scott Keeter, What Americans Know about Politics and Why It Matters (New 
Haven: Yale University Press, 1996); William A. Galston, “Political Knowledge, Political Engagement, and Civic 
Education,” Annual Review of Political Science 4 (2001): 217-34; and John Zaller, The Nature and Origins of Mass 
Opinion (New York: Cambridge University Press, 1992). 

12. Diane Ravitch, “The Controversy Over National History Standards,” Bulletin of the American Academy of Arts 
and Sciences 51, no. 3 (1998): 14-28. 

13. Rebecca M. Callahan and Chandra Muller, Coming of Political Age: American Schools and the Civic 
Development of Immigrant Youth (New York: Russell Sage Foundation, 2013); and Torney-Purta, Barber, and 
Wilkenfeld, “Latino Adolescents’ Civic Development in the United States.” 

14. Langton and Jennings, “Political Socialization and the High School Civics Curriculum in the United States.” 

15. The survey was conducted by telephone by Universal Survey Inc. Two-thirds of the numbers called were cell 
phones; the rest were landlines. Interviewing began one day after the 2012 presidential election and continued for 
six weeks. The sampling design ensured that there are at least 75 respondents from each state. There were also 
oversamples of African Americans and Hispanics. The data have been weighted to match the basic demographics of 


23 



the US population, according to the March 2012 Current Population Survey. For more details on the survey, see 
Kawashimi and Levine, “Policy Effects on Informed Political Engagement.” 

16. Respondents were asked to identify the state in which they went to high school. 

17. Center for Information & Research on Civic Learning and Engagement, “New CIRCLE Fact Sheet.” 

18. Carlson, “Out of the Classroom and Into the Voting Booth?” 

19. For example, only 56 percent of respondents who attended high school in states with, ostensibly, a civics 
assessment report that they took “a standardized test on civics or U.S. government . . . not designed by your own 
teacher but required by the district or state your school was in.” There are a variety of possible explanations for why 
this percentage is relatively low. It could have been noncompliance on the part of schools and districts, or 
respondents may not remember a nonconsequential test or do not realize or recall that a given test was required by 
the state. Center for Information & Research on Civic Learning and Engagement, “New CIRCLE Fact Sheet.” 

20. Delli Carpini and Keeter, What Americans Know about Politics and Why It Matters. 

21. Both educational attainment and books in the home are coded as a series of dummy variables for each category 
rather than as a single ordinal variable. 

22. The models employ weights to match the overall demographics with the national population (see Kawashimi and 
Levine, “Policy Effects on Informed Political Engagement”). To account for the risk of heteroskedasticity (in other 
words, for respondents being clustered into states), the model employs robust standard errors and accounts for 
clustering by state. The analysis has also been run using hierarchical (in other words, mixed) models, and the results 
are substantively similar. 

23. The figures in Appendix 1 include the 95 percent confidence intervals for each point estimate, which have been 
generated using the CLARIFY program in in STATA. See Michael Tomz, Jason Wittenberg, and Gary King, 
“CLARIFY : Software for Interpreting and Presenting Statistical Results,” Journal of Statistical Software 8, no. 1 
(2003). 

24. Note that the coefficient for age is negative, which means that older respondents have, on average, lower civic 
knowledge. While this runs counter to the expectation for civic knowledge (sophistication) in the general population, 
where greater age is associated with more, not less, knowledge, this result for a sample of 18-24-year-olds provides 
still more evidence that civics instruction in high school is indeed causally related to civic knowledge. If classroom 
instruction leads to more knowledge, we would expect it to fade as time passes or, in other words, as these 
respondents get older. (Other factors explain the greater civic knowledge of people past the age of 24.) 

25. When limiting the analysis to Asian American respondents, the effect is also large and statistically significant. 
However, Asian Americans are not included in the discussion because of their small sample size; they are correct as 
a proportion of the population, but small in absolute numbers (92 cases). Further research should examine the impact 
of civic education, and civics assessments specifically, on the Asian American population. Note that “immigrant” is 
operationalized as students who are either themselves immigrants to the US or who have at least one immigrant 
parent. 

26. Underscoring the magnitude of this effect among Hispanic immigrants, the difference between its magnitude and 
the effect size for the general population is statistically significant. This is determined by a model in which the 
coefficient for an interaction term between “Hispanic immigrant” and “consequential civics assessment (CCA)” is 
significantly different than the effect of a CCA on the general population. Similar interaction terms for African 
Americans, Hispanics, and immigrants are not significant. In other words, while a CCA has larger effects for each of 
these groups than the general population, the differences do not achieve the conventional threshold of statistical 
significance (p <.05). 

27. Note that roughly half (48 percent) of Hispanics are classified as immigrants. 

28. John B. Diamond, “Accountability Policy, School Organization, and Classroom Practice: Partial Recoupling and 
Educational Opportunity,” Education and Urban Society 44, no. 2 (2012): 151-82; and Cecilia Elena Rouse et al., 
“Feeling the Florida Heat? How Low-Performing Schools Respond to Voucher and Accountability Pressure,” 
American Economic Journal: Economic Policy 5, no. 2 (2013): 251-81. 

29. Anna Roefsky Saavedra, “Dry to Dynamic Civic Education Curricula,” in Making Civics Count, 135-60. 

30. Campbell, “Voice in the Classroom.” 

3 1 . Specifically, respondents were asked whether they had taken “a class called something like civics, American 
government, or government.” If they said no, they were asked “Did you take a course by a different name in which 
you spent a lot of time learning about politics, government, social issues, or law?” If they answered yes to either 
question, they were then asked whether the class had these features. The analysis combines responses from both 
groups into a single index. 


24 



32. Supovitz, “Can High Stakes Testing Leverage Educational Improvement?”; and Julian Vasquez Heilig and 
Linda Darling-Hammond, “Accountability Texas-Style: The Progress and Learning of Urban Minority Students in a 
High-Stakes Testing Context,” Educational Evaluation and Policy Analysis 30, no. 2 (2008): 75-110 

33. Meghan Condon, “Practice Makes Participants: How Communication Skills Acquired in School Affect Political 
Engagement” (dissertation, University of Wisconsin-Madison, 2012); and Donald P. Green et al., “Does Knowledge 
of Constitutional Principles Increase Support for Civil Liberties? Results from a Randomized Field Experiment,” 

The Journal of Politics 73, no. 2 (201 1): 463-76. 


25 



